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Imaging technologies such as dynamic viewpoint generation are engineered for 
incoherent radiation using the traditional light field, and for coherent radiation 
using electromagnetic field theory. We present a model of coherent image 
formation that strikes a balance between the utility of the light field and 
the comprehensive predictive power of Maxwell's equations. We synthesize 
research in optics and signal processing to formulate, capture, and form images 
from quasi light fields, which extend the light field from incoherent to coherent 
radiation. Our coherent cameras generalize the classic beamforming algorithm 
in sensor array processing, and invite further research on alternative notions 
of image formation. 

1 Introduction 

The light field represents radiance as a function of position and direction, thereby decom- 
posing optical power flow along rays. The light field is an important tool used in many 
imaging applications in different disciplines, but is traditionally limited to incoherent light. 
In computer graphics, a rendering pipeline can compute new views at arbitrary camera po- 
sitions from the light field p]. In computational photography, a camera can measure the 
light field and later generate images focused at different depths, after the picture is taken 
[2]. In electronic displays, an array of projectors can present multiple viewpoints encoded in 
the light field, enabling 3D television [3]. Many recent incoherent imaging innovations have 
been made possible by expressing image pixel values as appropriate integrals over hght field 
rays. 

For coherent imaging applications, the value of decomposing power by position and di- 
rection has long been recognized without the aid of a light field, since the complex-valued 
scalar field encodes direction in its phase. A hologram encodes multiple viewpoints, but in 
a different way than the light field [1]. An ultrasound machine generates images focused 
at different depths, but from air pressure instead of light field measurements [5]. A Wigner 
distribution function models the operation of optical systems in simple ways, by conveniently 



inferring direction from the scalar field instead of computing non-negative light field values 
[6J. Comparing these applications, coherent imaging uses the scalar field to achieve results 
similar to those that incoherent imaging obtains with the light field. 

Our goal is to provide a model of coherent image formation that combines the utility of 
the light field with the comprehensive predictive power of the scalar field. The similarities 
between coherent and incoherent imaging motivate exploring how the scalar field and light 
field are related, which we address by synthesizing research across three different communi- 
ties. Each community is concerned with a particular Fourier transform pair and has its own 
name for the light field. In optics, the pair is position and direction, and Walther discovered 
the first generalized radiance function by matching power predictions made with radiometry 
and scalar field theory [Tj. In quantum physics, the pair is position and momentum, and 
Wigner discovered the first quasi-probability distribution, or phase-space distribution, as an 
aid to computing the expectation value of a quantum operator In signal processing, 
the pair is time and frequency, and while instantaneous spectra were used as early as 1890 
by Sommerfeld, Ville is generally credited with discovering the first nontrivial quadratic 
time-frequency distribution by considering how to distribute the energy of a signal over time 
and frequency [H] . Walther, Wigner, and Ville independently arrived at essentially the same 
function, which is one of the ways to express a light field for coherent radiation in terms of 
the scalar field. 

The light field has its roots in radiometry, a phenomenological theory of radiative power 
transport that began with Herschel's observations of the sun [10], developed through the 
work of astrophysicists such as Chandrasekhar jTT], and culminated with its grounding in 
electromagnetic field theory by Friberg et al. ^L2\. The light field represents radiance, 
which is the fundamental quantity in radiometry, defined as power per unit projected area 
per unit solid angle. Illuminating engineers would integrate radiance to compute power 
quantities, although no one could vahdate these calculations with the electromagnetic field 
theory formulated by Maxwell. Gershun was one of many physicists who attempted to 
physically justify radiometry, and who introduced the phrase light field to represent a three- 
dimensional vector field analogous to the electric and magnetic fields [13j. Gershun's light 
field is a degenerate version of the one we discuss, and more closely resembles the time- 
averaged Poynting vector that appears in a rigorous derivation of geometric optics [14] . 
Subsequently, Walther generalized radiometry to coherent radiation in two different ways [71 
\T5\ . and Wolf connected Walther's work to quantum physics [IH], ultimately leading to 
the discovery of many more generalized radiance functions [T7j and a firm foundation for 
radiometry |12j . 

Meanwhile, machine vision researchers desired a representation for all the possible pic- 
tures a pinhole camera might take in space-time, which led to the current formulation of the 
light field. Inspired by Leonardo da Vinci, Adelson and Bergen defined a plenoptic function 
to describe "everything that can be seen" as the intensity recorded by a pinhole camera, 
parametrized by position, direction, time, and wavelength [18]. Levoy and Hanrahan tied 
the plenoptic function more firmly to radiometry, by redefining Gershun's phrase light field 
to mean radiance parametrized by position and direction [1]. Gortler et al. introduced the 
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same construct, but instead called it the lumigraph [19]. The light field is now the dominant 
terminology used in incoherent imaging contexts. 

Our contribution is to describe and characterize all the ways to extend the light field to 
coherent radiation, and to interpret coherent image formation using the resulting extended 
light fields. We call our extended light fields quasi light fields, which are analogous to the 
generalized radiance functions of optics, the quasi-probability and phase-space distributions 
of quantum physics, and the quadratic class of time-frequency distributions of signal pro- 
cessing. Agarwal et al. have already extended the light field to coherent radiation fTT], and 
the signal processing community has already classified all of the ways to distribute power 
over time and frequency [20]. Both have traced their roots to quantum physics. But to our 
knowledge, no one has connected the research to show 1) that the quasi light fields represent 
all the ways to extend the light field to coherent radiation, and 2) that the signal processing 
classification informs which quasi light field to use for a specific application. We further 
contextualize the references, making any unfamiliar literature more accessible to specialists 
in other areas. 

Our paper is organized as follows. We describe the traditional light field in Section [2l We 
formulate the quasi light fields in Section [31 by reviewing and relating the relevant research 
in optics, quantum physics, and signal processing. In Section |U we describe how to capture 
quasi light fields, discuss practical sampling issues, and illustrate the impact of light field 
choice on energy localization. In Section [5l we describe how to form images with quasi light 
fields. We derive a light field camera, demonstrate and compensate for diffraction limitations 
in the near zone, and generalize the classic beamforming algorithm in sensor array processing. 
We conclude the paper in Section [6], where we remark on the utility of quasi light fields and 
future perspectives on image formation. 



The light field is a useful tool for incoherent imaging because it acts as an intermediary 
between the camera and the picture, decoupling information capture and image production: 
the camera measures the light field, from which many different traditional pictures can be 
computed. We define a pixel in the image of a scene by a surface patch a and a virtual 
aperture (Figure [T]). Specifically, we define the pixel value as the power P radiated by 
(T towards the aperture, just as an ideal single- lens camera would measure. According to 
radiometry, P is an integral over a bundle of light field rays [21] : 



where L(r, s) is the radiance at position r and in unit direction s, ip is the angle that s makes 
with the surface normal at r, and Qr is the solid angle subtended by the virtual aperture at 
r. The images produced by many different conventional cameras can be computed from the 
light field using [22]. 




2 The Traditional Light Field 
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Figure 1: We can compute the value of each pixel in an image produced by an arbitrary virtual 
camera, defined as the power emitted from a scene surface patch towards a virtual aperture, by 
integrating an appropriate bundle of light field rays that have been previously captured with remote 
hardware. 

The light field has an important property that allows us to measure it remotely: the 
light field is constant along rays in a lossless medium [21]. To measure the light field on 
the surface of a scene, we follow the rays for the images we are interested in, and intercept 
those rays with our camera hardware (Figured]). However, our hardware must be capable 
of measuring the radiance at a point and in a specific direction; a conventional camera that 
simply measures the irradiance at a point is insufficient. We can discern directional power 
fiow using a lens array, as is done in a plenoptic camera [2|. 

In order to generate coherent images using the same framework described above, we must 
overcome three challenges. First, we must determine how to measure power fiow by position 
and direction to formulate a coherent light field. Second, we must capture the coherent light 
field remotely and be able to infer behavior at the scene surface. Third, we must be able to 
use (PP) to produce correct power values, so that we can form images by integrating over the 
coherent light field. We address each challenge in a subsequent section. 



We motivate, systematically generate, and characterize the quasi light fields by relating 
existing research. We begin in Section [STT] with research in optics that frames the challenge 
of extending the light field to coherent radiation in terms of satisfying a power constraint 
required for radiometry to make power predictions consistent with scalar field theory. While 
useful in developing an intuition for quasi light fields, the power constraint does not allow 
us to easily determine the quasi light fields. We therefore proceed in Section 13.21 to describe 
research in quantum physics that systematically generates quasi light fields satisfying the 
power constraint, and that shows how the quasi light fields are true extensions that reduce 
to the traditional light field under certain conditions. While useful for generating the quasi 
light fields, the quantum physics approach does not allow us to easily characterize them. 
Therefore, in Section 13.31 we map the generated quasi light fields to the quadratic class of 



3 Formulating Quasi Light Fields 
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time-frequency distributions, which has been extensively characterized and classified by the 
signal processing community. By relating research in optics, quantum physics, and signal 
processing, we express all the ways to extend the light field to coherent radiation, and provide 
insight on how to select an appropriate quasi light field for a particular application. 

We assume a perfectly coherent complex scalar field U{r) at a fixed frequency u for 
simplicity, although we comment in Section [6] on how to extend the results to broadband, 
partially coherent radiation. The radiometric theory we discuss assumes a planar source at 
z = 0. Consequently, although the light field is defined in three-dimensional space, much of 
our analysis is confined to planes z = Zq parallel to the source. Therefore, for convenience, 
we use r = (x, y, z) and s = (s^,, s^, Sz) to indicate three-dimensional vectors and = (x, y) 
and s±_ = {sr^, Sy) to indicate two-dimensional, projected versions. 

3.1 Intuition from Optics 

An extended light field must produce accurate power transport predictions consistent with 
rigorous theory; thus the power computed from the scalar field using wave optics determines 
the allowable light fields via the laws of radiometry. One way to find extended light fields 
is to guess a light field equation that satisfies this power constraint, which is how Walther 
identified the first extended light field [7]. The scenario involves a planar source at z = 
described by f/(r), and a sphere of large radius p centered at the origin. We use scalar 
field theory to compute the fiux through part of the sphere, and then use the definition of 
radiance to determine the light field from the fiux. 

According to scalar field theory, the differential flux d<l> through a portion of the sphere 
subtending differential solid angle dfl is given by integrating the radial component of the 
energy flux density vector F. From diffraction theory, the scalar fleld in the far zone is 




(2) 



where /c = 27r/A is the wave number, A is the wavelength, and 





is the plane wave component in direction s [23]. Now 




(4) 



so that 




(5) 



According to radiometry, radiant intensity is flux per unit solid angle 




(6) 
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Radiance is /(s) per unit projected area [21], and this is where the guessing happens: there 
are many ways to distribute (jSD over projected area by factoring out and an outer integral 
over the source plane, but none yield light fields that satisfy all the traditional properties of 
radiance |2l]. One way to factor is to substitute the expression for a(s) from ([3]) into ([6]) 
and change variables: 



/(s) = s^j s^ju[v+ ^r'^ U* [v - ^r'^ exp {-iks^ ■ r^) dV 

The bracketed expression is Walther's first extended light field 



dV. 



(7) 
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r' ) exp (— z27rs_L ■ r'^) d^r' 



(9) 



is the Wigner distribution [25]. We may manually factor ([6]) differently to obtain other 
extended light fields in an ad hoc manner, but it is hard to find and verify the properties of 
all extended light fields this way, and we would have to individually analyze each light field 
that we do manage to find. So instead, we pursue a systematic approach to exhaustively 
identify and characterize the extended light fields that guarantee the correct radiant intensity 
in (ED. 



3.2 Generating Explicit Extensions from Quantum Physics 

The mathematics of quantum physics provides us with a systematic extended light field 
generator that factors the radiant intensity in ([6]) in a structured way. Walther's extended 
light field in (JHl) provides the hint for this connection between radiometry and quantum 
physics. Specifically, Wolf recognized the similarity between Walther's light field and the 
Wigner phase-space distribution [8] from quantum physics [16]. Subsequently, Agarwal, 
Foley, and Wolf repurposed the mathematics behind phase-space representation theory to 
generate new light fields instead of distributions [T7j. We summarize their approach, define 
the class of quasi light fields, describe how quasi light fields extend traditional radiometry, 
and show how quasi light fields can be conveniently expressed as filtered Wigner distributions. 

Agarwal et al.'s key insight was to introduce a position operator r_L and a direction 
operator Sj_ that obey the commutation relations [26] 

[x, s^] = i\/27i, [y, Sy] = iX/27i, (10) 

and to map the different ways of ordering the operators to different extended light fields. This 
formulation is valuable for two reasons. First, flTUl) is analogous to the quantum- mechanical 
relations for position and momentum, allowing us to exploit the phase-space distribution 
generator from quantum physics for our own purposes, thereby providing an explicit formula 
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for extended light fields. Second, in the geometric optics limit as A 0, the operators 
commute per (ITUI) . so that all of the extended light fields collapse to the same function that 
can be related to the traditional light field. Therefore, Agarwal et al.'s formulation not only 
provides us with different ways of expressing the light field for coherent radiation, but also 
explains how these differences arise as the wavelength becomes non-negligible. 

We now summarize the phase-space representation calculus that Agarwal and Wolf in- 
vented [27] to map operator orderings to functions, which Agarwal et al. later applied to 
radiometry [T7], culminating in a formula for extended light fields. The phase-space repre- 
sentation theory generates a function from any operator L for each distinct way Q of 
ordering collections of and s^. So by choosing a specific L defined by its matrix elements 
using the Dirac notation [26] 



L|r5)=f/(r«)f/*(rC), (11) 



and supplying L as input, we obtain the extended light fields 

L^(r,s)= (^Ay.,L^(r^,s^) (12) 

as outputs. The power constraint from Section 13.11 translates to a minor constraint on the 
allowed orderings Q, so that can be factored from ([6]). Finally, there is an explicit formula 
for [27], which in Friberg et al.'s form [12] reads 



L^ir, s) = j^s, jjj n (u, kr'[_) exp [-tu ■ (r^ - r'^)] exp {-iks^ ■ r'_[) 

xu(^r' + ir"^ U* (^r' - ^r"^ d\ d V dV", (13) 

where is a functional representation of the ordering Q. 

Previous research has related the extended light fields to the traditional light field, 
by examining how the behave for globally incoherent light of a small wavelength, an 
environment technically modeled by a quasi-homogeneous source in the geometric optics 
limit where A — 0. As A ^ 0, r_L and commute per (fTOj) . so that all orderings Q are 
equivalent and all of the extended light fields collapse to the same function. Since, in the 
source plane, Foley and Wolf showed that one of those light fields behaves like traditional 
radiance [28j for globally incoherent light of a small wavelength, all of the behave like 
traditional radiance for globally incoherent light of a small wavelength. Furthermore, Friberg 
et al. showed that many of the are constant along rays, for globally incoherent light of 
a small wavelength [12j. The thereby subsume the traditional light field, and globally 
incoherent light of a small wavelength is the environment in which traditional radiometry 
holds. 

To more easily relate to the signal processing literature, we conveniently express 
as a filtered Wigner distribution. We introduce a function 11 and substitute 

r2(u,v)= // n(-a,-b)exp[-i(a-u + b-v)] d^ad^fo (14) 
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into (fT3|) . integrate first over u, then over a, and finally substitute b = s'j_ — s^: 
L^(r, s) = (^A^ \,JJ n(r^ - r'^, - s'^)2n(r', s'JX) dV d's' 

^ s,n(rx,s^)®2n(r,sx/A). (15) 

The symbol ® in (fT5|) denotes convolution in both and s^. Each filter kernel 11 yields a 
different light field. There are only minor restrictions on 11, or equivalently on Cl. Specifically, 
Agarwal and Wolf's calculus requires that |27j 

1/Q is an entire analytic function with no zeros on the real component axes. (16) 

Agarwal et al.'s derivation additionally requires that 

fi(0, v) = 1 for all V, (17) 

so that satisfies the laws of radiometry and is consistent with [T7] . 

We call the functions L^, the restricted class of extended light fields that we have system- 
atically generated, quasi light fields, in recognition of their connection with quasi-probability 
distributions in quantum physics. 




3.3 Characterization from Signal Processing 

Although we have identified the quasi light fields and justified how they extend the traditional 
light field, we must still show that we have found all possible ways to extend the light field 
to coherent radiation, and we must indicate how to select a quasi light field for a specific 
application. We address both concerns by relating quasi light fields to bilinear forms of 
U and U* that are parameterized by position and direction. First, such bilinear forms 
refiect all the different ways to represent the energy distribution of a complex signal in signal 
processing, and therefore contain all possible extended light fields, allowing us to identify any 
unaccounted for by quasi light fields. Second, we may use the signal processing classification 
of bilinear forms to characterize quasi light fields and guide the selection of one for an 
application. 

To relate quasi light fields to bilinear forms, we must express the filtered Wigner distri- 
bution in f|T5|) as a bilinear form. Towards this end, we first express the filter kernel 11 in 
terms of another function K: 

n(a, h)= j K l^-a + ^v, -a - ^v^ exp(-i27rb • v) d\. (18) 

We substitute ( ITSll into ( fTSi) . integrate first over s'^, then over v, and finally substitute 

= r' + ir", = r' - (19) 
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to express the quasi light field as 

L{r,s) = (Ay..||f/(r^){ir(r5-r^,r5-r^) 

X exp [-tks^ ■ (r^ - rg] } U* (r^) dV^dV^. (20) 

We recognize that (1201) is a bilinear form of U and U*, with kernel indicated by the braces. 

The structure of the kernel of the bilinear form in fl2U]) limits L to a shift-invariant 
energy distribution. Specifically, translating the scalar field in fl20|) in position and direction 
orthogonal to the 2;-axis according to 

f/(r) ^ f/ (r - r°) exp (zA;s^ ■ r^) (21) 

results in a corresponding translation in position and direction in the light field, after rear- 
ranging terms: 

L(r, s) -> L (r - r°, s - s°) . (22) 

Such shift-invariant bilinear forms comprise the quadratic class of time-frequency distribu- 
tions, which is sometimes misleadingly referred to as Cohen's class |20] . 

The quasi light fields represent all possible ways of extending the light field to coherent 
radiation. This is because any reasonably defined extended light field must be shift-invariant 
in position and direction, as translating and rotating coordinates should modify the scalar 
field and light field representations in corresponding ways. Thus, on the one hand, an 
extended light field must be a quadratic time-frequency distribution. On the other hand, fl2Ul) 
implies that quasi light fields span the entire class of quadratic time-frequency distributions, 
apart from the constraints on 11 described at the end of Section 13.21 The constraint in 
( IT7|) is necessary to satisfy the power constraint in ([6]), which any extended light field must 
satisfy. The remaining constraints in (fT6|) are technical details concerning analyticity and 
the location of zeros; extended light fields strictly need not satisfy these mild constraints, 
but the light fields that are ruled out are well-approximated by light fields that satisfy them. 

We obtain a concrete sensor array processing interpretation of quasi light fields by group- 
ing the exponentials in fl20|) with U instead of K: 

x|f/(r^)exp [tks- (r - r^)] }*dV^dV^. (23) 

The integral in fl23|) is the expected value of the energy of the output of a spatial filter 
with impulse response exp{iks ■ r) applied to the scalar field, when using K to estimate the 
correlation E[U{r^)U*{r'~^)] by 

f/(rgir(r^-r^,r5-rgf/*(rg. (24) 

That is, the choice of quasi light field corresponds to a choice of how to infer coherence 
structure from scalar field measurements. In adaptive beamforming, the spatial filter exp(i/cs- 
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r) focuses a sensor array on a particular plane wave component, and K serves a similar role 
as the covariance matrix taper that gives rise to design features such as diagonal loading [2S] . 
But for our purposes, the sensor array processing interpretation in (12^ allows us to cleanly 
separate the choice of quasi light field in K from the plane wave focusing in the exponentials. 

Several signal processing books meticulously classify the quadratic class of time-frequency 
distributions by their properties, and discuss distribution design and use for various applica- 
tions [23 ED]- We can use these resources to design quasi light fields for specific applications. 
For example, if we desire a light field with fine directional localization, we may first try the 
Wigner quasi light field in ([8]), which is a popular starting choice. We may then discover 
that we have too many artifacts from interfering spatial frequencies, called cross terms, and 
therefore wish to consider a reduced interference quasi light field. We might try the modified 
B-distribution, which is a particular reduced interference quasi hght field that has a tunable 
parameter to suppress interference. Or, we may decide to design our own quasi light field in 
a transformed domain using ambiguity functions. The resulting tradeoffs can be tailoring to 
specific application requirements. 

4 Capturing Quasi Light Fields 

To capture an arbitrary quasi light field, we sample and process the scalar field. In inco- 
herent imaging, traditional light fields are typically captured by instead making intensity 
measurements at a discrete set of positions and directions, as in done in the plenoptic cam- 
era p]. While it is possible to apply the same technique to coherent imaging, only a small 
subset of quasi light fields that exhibit poor localization properties can be captured this way. 
In comparison, all quasi light fields can be computed from the scalar field, as in (fTSll . We 
therefore sample the scalar field with a discrete set of sensors placed at different positions in 
space, and subsequently process the scalar field measurements to compute the desired quasi 
light field. We describe the capture process for three specific quasi light fields in Section 14. 
and demonstrate the different localization properties of these quasi light fields via simulation 
in Section Wf2\ 

4.1 Sampling the Scalar Field 

To make the capture process concrete, we capture three different quasi light fields. For 
simplicity, we consider a two-dimensional scene and sample the scalar field with a linear 
array of sensors regularly spaced along the ?/-axis (Figure [2]). With this geometry, the 
scalar field U is parameterized by a single position variable y, and the discrete light field 
^ is parameterized by y and the direction component Sy. The sensor spacing is which 
we assume is fine enough to ignore aliasing effects. This assumption is practical for long- 
wavelength applications such as millimeter-wave radar. For other applications, aliasing can 
be avoided by applying an appropriate pre-filter. From the sensor measurements, we compute 
three different quasi light fields, including the spectrogram and the Wigner. 

Although the spectrogram quasi light field is attractive because it can be captured like a 
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Figure 2: We capture a discrete quasi light field i by sampling the scalar field at regularly-spaced 
sensors and processing the resulting measurements. We may optionally apply an aperture stop T 
to mimic traditional light field capture, but this restricts us to capturing quasi light fields with 
poor localization properties. 



traditional light field by making intensity measurements, it exhibits poor localization prop- 
erties. Zhang and Levoy explain [30] how to capture the spectrogram by placing an aperture 
stop specified by a transmission function T over the desired position y before computing 
a Fourier transform to extract the plane wave component in the desired direction Sy, and 
previously Ziegler et al. used the spectrogram as a coherent light field to represent a holo- 
gram [4j. The spectrogram is an important quasi light field because it is the building block 
for the quasi light fields that can be directly captured by making intensity measurements, 
since all non-negative quadratic time-frequency distributions, and therefore all non-negative 
quasi light fields, are sums of spectrograms [20]. Ignoring constants and s^, we compute the 
discrete spectrogram from the scalar field samples by 



yj 



T{nd)U{y + nd) exp {—ikndsy) 



(25) 



The Wigner quasi light field is a popular choice that exhibits good energy localization 
in position and direction [20]. We already identified the Wigner quasi light field in ([8]); the 



discrete version is 

t^(^y^ Sy) = ^u{y + nd/2)U*{y - nd/2) exp {-ikndSy) . (26) 

n 

Evidently, the spectrogram and Wigner distribute energy over position and direction in very 
different ways. Per fl25l) . the spectrogram first uses a Fourier transform to extract directional 
information and then computes a quadratic energy quantity, while the Wigner does the 
reverse, per (|26|) . On the one hand, this reversal allows the Wigner to better localize energy 
in position and direction, since the Wigner is not bound by the classical Fourier uncertainty 
principle as the spectrogram is. On the other hand, the Wigner's nonlinearities introduce 
cross-term artifacts by coupling energy in different directions, thereby replacing the simple 
uncertainty principle with a more complicated set of tradeoffs |20] . 
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We now introduce a third quasi light field to capture, in order to help us understand the 
implications of requiring quasi light fields to exhibit traditional light field properties. Specif- 
ically, traditional light fields have real non-negative values that are zero where the scalar 
field is zero, whereas no quasi light field behaves this way |23]. Although the spectrogram 
has non-negative values, the support of both the spectrogram and Wigner spills over into 
regions where the scalar field is zero. In contrast, the conjugate Rihaczek quasi light field, 
which can be obtained by substituting for a*(s) in and factoring, is identically zero 
at all positions where the scalar field is zero and for all directions in which the plane wave 
component is zero: 

L^(r, s) = s^f/*(r) exp{iks ■ r)a(s). (27) 

However, unlike the non-negative spectrogram and the real Wigner, the Rihaczek is complex- 
valued, as each of its discoverers independently observed: Walther in optics [I5], Kirkwood 
in quantum physics [31], and Rihaczek in signal processing [32]. The discrete conjugate 
Rihaczek quasi light field is 

i^{y, Sy) = U*{y) exp {ikysy) U {nd) exp {—ikndSy) . (28) 

n 

4.2 Localization Tradeoffs 

Different quasi light fields localize energy in position and direction in different ways, so 
that the choice of quasi light field impacts the potential resolution achieved in an imaging 
application. We illustrate the diversity of behavior by simulating a plane wave propagating 
past a screen edge and computing the spectrogram, Wigner, and Rihaczek quasi light fields 
from scalar field samples (Figure [3]). This simple scenario stresses the main tension between 
localization in position and direction: each quasi light field must encode the position of the 
screen edge as well as the downward direction of the plane wave. The quasi light fields serve 
as intermediate representations used to jointly estimate the position of the screen edge and 
the orientation of the plane wave. 

Our simulation accurately models diffraction using our implementation of the angular 
spectrum propagation method, which is the same technique used in commercial optics soft- 
ware to accurately simulate wave propagation [33|. We propagate a plane wave with wave- 
length A = 3 mm a distance i? = 50 m past the screen edge, where we measure the scalar 
field and compute the three discrete light fields using (l25l) . (1261) . and (l28ll . To compute 
the light fields, we set d = A/10, run the summations over \n\ < 10/A, and use a rect- 
angular window function of width 10 cm for T. We plot i^, and in terms of 
the two-plane parameterization of the light field pT], so that each ray is directed from a 
point u in the plane of the screen towards a point y in the measurement plane, and so that 
Sy = iy-u)/[R' + iy-u)'f\ 

We compare each light field's ability to estimate the position of the screen edge and the 
orientation of the plane wave (Figure [3]) • Geometric optics provides an ideal estimate: we 
should ideally only see rays pointing straight down {u = y) past the screen edge, correspond- 
ing to a diagonal line in the upper-right quadrant of the light field plots. Instead, we see 
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Figure 3: The spectrogram does not resolve a plane wave propagating past the edge of an opaque 
screen as well as other quasi light fields, such as the Wigner and Rihaczek. We capture all three quasi 
light fields by sampling the scalar field with sensors and processing the measurements according to 
([25]) , ([25]) , and ([25]) . The ringing and blurring in the light field plots indicate the diffraction fringes 
and energy localization limitations. 



blurred lines with ringing. The ringing is physically accurate and indicates the diffraction 
fringes formed on the measurement plane. The blurring indicates localization limitations. 
While the spectrogram's window T can be chosen to narrowly localize energy in either po- 
sition or direction, the Wigner narrowly localizes energy in both, depicting instantaneous 
frequency without being limited by the classical Fourier uncertainty principle [20] . 

It may seem that the Wigner light field is preferable to the others and the clear choice for 
all applications. While the Wigner light field possesses excellent localization properties, it 
exhibits cross-term artifacts due to interference from different plane wave components. An 
alternative quasi light field such as the Rihaczek can strike a balance between localization 
and cross-term artifacts, and therefore may be a more appropriate choice, as discussed at 
the end of Section [X^ If our goal were to only estimate the position of the screen edge, we 
might prefer the spectrogram; to jointly estimate both position and plane wave orientation, 
we prefer the Wigner; and if there were two plane waves instead of one, we might prefer 
the Rihaczek. One thing is certain, however: we must abandon non-negative quasi light 
fields to achieve better localization tradeoffs, as all non-negative quadratic time-frequency 
distributions are sums of spectrograms and hence exhibit poor localization tradeoffs pUj . 
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5 Image Formation 



We wish to form images from quasi light fields for coherent applications similarly to how we 
form images from traditional light fields for incoherent applications, by using ([1]) to integrate 
bundles of light field rays to compute pixel values (Figured]). However, simply selecting a 
particular captured quasi light field L and evaluating ([T]) raises three questions about the 
validity of the resulting image. First, is it meaningful to distribute coherent energy over 
surface area by factoring radiant intensity in ([6])? Second, does the far-zone assumption 
implicit in radiometry and formalized in ([2]) limit the applicability of quasi field fields? And 
third, how do we capture quasi light field rays remotely if, unlike the traditional light field, 
quasi light fields need not be constant along rays? 

The first question is a semantic one. For incoherent light of a small wavelength, we 
define an image in terms of the power radiating from a scene surface towards an aperture, 
and physics tells us that this uniquely specifies the image (Section[3]), which may be expressed 
in terms of the traditional light field. If we attempt to generalize the same definition of an 
image to partially coherent, broadband light, and specifically to coherent light at a non- 
zero wavelength, we must ask how to isolate the power from a surface patch towards the 
aperture, according to classical wave optics. But there is no unique answer; different isolation 
techniques correspond to different quasi light fields. Therefore, to be well-defined, we must 
extend the definition of an image for coherent light to include a particular choice of quasi 
light field, which corresponds to a particular factorization of radiant intensity. 

The second and third questions speak of assumptions in the formulation of quasi light 
fields and in the image formation from quasi light fields, which can lead to coherent imaging 
inaccuracies when these assumptions are not valid. Specifically, unless the scene surface and 
aperture are far apart, the far-zone assumption in ([2]) does not hold, so that quasi light fields 
are incapable of modeling near- zone behavior. Also, unless we choose a quasi light field that 
is constant along rays, such as an angle-impact Wigner function [3l], remote measurements 
might not accurately refiect the light field at the scene surface [35], resulting in imaging 
inaccuracies. Therefore, in general, integrating bundles of remotely captured quasi light field 
rays produces an approximation of the image we have defined. We assess this approximation 
by building an accurate near-zone model in Section 15. ![ simulating imaging performance of 
several coherent cameras in Section 15. 2^ and showing how our image formation procedure 
generalizes the classic beamforming algorithm in Section 15. 3[ 

5.1 Near-Zone Radiometry 

We take a new approach to formulating light fields for coherent radiation that avoids making 
the assumptions that 1) the measurement plane is far from the scene surface and 2) light 
fields are constant along rays. The resulting light fields are accurate in the near zone, and 
may be compared with quasi light fields to understand quasi light field limitations. The key 
idea is to express a near-zone light field L(r, s) on the measurement plane in terms of the 
infinitesimal fiux at the point where the line containing the ray (r, s) intersects the scene 
surface (Figure S]). First we compute the scalar field at the scene surface, next we compute 
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the infinitesimal flux, and then we identify a light field that predicts the same flux using the 
laws of radiometry. In contrast with Walther's approach (Section 13. ip . 1) we do not make 
the far-zone approximation as in ([2]), and 2) we formulate the light field in the measurement 
plane instead of in the source plane at the scene surface. Therefore, in forming an image 
from a near-zone light field, we are not limited to the far zone and we need not relate the 
light field at the measurement plane to the light field at the scene surface. 




Figure 4: To ensure that integrating bundles of remote light field rays in the near zone results in 
an accurate image, we derive a light field L^{r,s) in the measurement plane from the infinitesimal 
flux d$ at the point where the ray originates from the scene surface patch. We thereby avoid 
making the assumptions that the measurement plane is far from the scene and that the light field 
is constant along rays. 

The first step in deriving a near- zone light field L for the ray (r, s) is to use the scalar 
field on the measurement plane to compute the scalar field at the point where the line 
containing the ray intersects the scene surface. We choose coordinates so that the measure- 
ment plane is the xy-plane, the scene lies many wavelengths away in the negative z < 
half-space, and r is at the origin. We denote the distance between the source on the scene 
surface and the point of observation r by p. Under a reasonable bandwidth assumption, the 
inverse diffraction formula expresses the scalar field at in terms of the scalar field on the 
measurement plane [36] : 

U{r^) = ^ / C'(r")^f^^^^^^=?^^^dV- (29) 

Next, we compute the differential flux d$ through a portion of a sphere at subtending 
differential solid angle dQ. We obtain d<l> by integrating the radial component of the energy 
flux density vector 

'dU*^^ dU. 



F(rP) = \- 



(30) 



To keep the calculation simple, we ignore amplitude decay across the measurement plane, 
approximating 

|rP-r^|^|rP| (31) 



15 



outside the exponential in (129|) . and 

d 



d\r 



PI 



r^-r^^l^l, (32) 



when evaluating ( 1301) . resulting in 



— \ a{-ps)a (-ps)- 



F(-ps) = ( — ) S(-ps)a*(-ps)^s, (33) 



where 



Thus, 



a(-ps) = ( ^ ) / f/(r'') exp(-«A;| - ps - r^|) dV^. (34) 



d$=(^— J s^a(-ps)r(-ps)dn. (35) 

Finally, we factor out Sz and an outer integral over surface area from d$/dfi to determine 
a near-zone light field. Unlike in Section 13.11 the nonlinear exponential argument in d 
complicates the factoring. Nonetheless, we obtain a near- zone light field that generalizes the 
Rihaczek by substituting ( !34l) for d* in ( l35l) . After factoring and freeing r from the origin 
by substituting r — ps for — ps, we obtain 

L^{r,s) = SzU*{r) exp{ikp)d{r — ps) 

SzU*{r) exp{tkp) j U (r^) exp {-ik\v - ps - r^|) dV^, (36) 

where the subscript p reminds us of this near-zone light field's dependence on distance. 

is evidently neither the traditional light field nor a quasi light field, as it depends 
directly on the scene geometry through an additional distance parameter. This distance 
parameter p is a function of r, s, and the geometry of the scene; it is the distance along s 
between the scene surface and r. We may integrate over a bundle of rays to compute the 
image pixel values just like any other light field, as long as we supply the right value of p 
for each ray. In contrast, quasi light fields are incapable of modeling optical propagation in 
the near zone, as it is insufficient to specify power flow along rays. We must also know the 
distance between the source and point of measurement along each ray. 

We can obtain near-zone generalizations of all quasi light fields through the sensor array 
processing interpretation in Section 13.31 Recall that each quasi light field corresponds to a 
particular choice of the function K in (1231) . For example, setting /^(a, b) = 5(b), where 5 
is the Dirac delta function, yields the Rihaczek quasi light field in fl27j) . To generalize 
quasi light fields to the near zone, we focus at a point instead of a plane wave component by 
using a spatial filter with impulse response exp {—ik |r — ps|) instead of exp(iA;s ■ r) in fl2^ . 
Then, choosing -ft'(a, b) = 5(b) yields L^, the near- zone generalization of the Rihaczek in 
( !36|) . and choosing other functions K yield near-zone generalizations of the other quasi light 
fields. 
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5.2 Near-Zone Diffraction Limitations 



We compute and compare image pixel values using the Rihaczek quasi light field and 
its near-zone generalization L^, demonstrating how all quasi light fields implicitly make the 
Fraunhofer diffraction approximation that limits accurate imaging to the far zone. First, we 
construct coherent cameras from and L^. For simplicity, we consider a two-dimensional 
scene and sample the light fields, approximating the integral over a bundle of rays (Figure 
d]) by the summation of discrete rays directed from the center of the scene surface patch 
to each sensor on a virtual aperture of diameter A, equally spaced every distance d in the 
measurement plane (Figure [5^). Ignoring constants and Sz, we compute the pixel values for 
a far-zone camera from the Rihaczek quasi light field in (I^T 



pR= J2 [U{nd)exp{-iknds'^)]" 

\nd\<A/2 



U{rad) exp (^—ikmdSy) 



(37) 



and for a near-zone camera from the near- zone generalization of the Rihaczek in (136!) . 



U{nd) exp (— zfcA„) 

\nd\<A/2 



exp (— ifcA^ 



(38) 



In (1371) . denotes the unit direction from to the rfi^ sensor, and in (1381) . A„ denotes the 
distance between and the rfi^ sensor. 

By comparing the exponentials in (1371) with those in (!38|) . we see that the near-zone 
camera aligns the sensor measurements along spherical wavefronts diverging from the point 
of focus r^, while the far-zone camera aligns measurements along plane wavefront approxi- 
mations (Figure [5b)- Spherical wavefront alignment makes physical sense in accordance with 
the Huygens-Fresnel principle of diffraction, while approximating spherical wavefronts with 
plane wavefronts is reminiscent of Fraunhofer diffraction. In fact, the far- zone approxima- 
tion in ([2]) used to derive quasi light fields follows directly from the Rayleigh-Sommerfeld 
diffraction integral by linearizing the exponentials, which is precisely Fraunhofer diffraction. 
Therefore, all quasi light fields are only valid for small Fresnel numbers, when the source 
and point of measurement are sufficiently far away from each other. 

We expect the near-zone camera to outperform the far- zone camera in near-zone imaging 
applications, which we demonstrate by comparing their ability to resolve small targets mov- 
ing past their field of view. As a baseline, we introduce a third camera with non-negative 
pixel values by restricting the summation over m in (1381) to \md\ < yl/2, which results 
in the beamformer camera used in sensor array processing [5l|37j. Alternatively, we could 
extend the summation over n in fl38|) to the entire array, but this would average anisotropic 
responses over a wider aperture diameter, resulting in a different image. We simulate an 
opaque screen containing a pinhole that is backlit with a coherent plane wave (Figure E]). 
The sensor array is -D = 2 m wide and just R = 1 m. away from the screen. The virtual 
aperture is A = 10 cm wide and the camera is focused on a fixed 1 mm pixel straight ahead 
on the screen. The pinhole has width 1 mm, which is smaller than the wavelength A = 3 
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Figure 5: The near-zone light field results in a camera that aligns spherical wavefronts diverging 
from the point of focus r^, in accordance with the Huygens-Fresnel principle of diffraction, while 
quasi light fields result in cameras that align plane wavefront approximations, in accordance with 
Fraunhofer diffraction. Quasi light fields are therefore only accurate in the far zone. We derive both 
cameras by approximating the integral over a bundle of rays by the summation of discrete light field 
rays (a), and we interpret the operation of each camera by how they align sensor measurements 
along wavefronts from (b). 

mm, so the plane wavefronts bend into slightly spherical shapes via diffraction. We move 
the pinhole to the right, recording pixel values \P^\, \P^\, and Pf for each camera at each 
pinhole position. Due to the nature of the coherent combination of the sensor measurements 
that produces the pixel values, each camera records a multi-lobed response. The width of 
the main lobe indicates the near-zone resolution of the camera. 

The near-zone camera is able to resolve the pinhole down to its actual size of 1 mm, greatly 
outperforming the far-zone camera which records a blur 66 cm wide, and even outperforming 
the beamformer camera. Neither comparison is surprising. First, with a Fresnel number of 
D^/RX f» 1333, the Fraunhofer approximation implicitly made by quasi light fields does not 
hold for this scenario, so we expect the far-zone camera to exhibit poor resolution. Second, 
the near- zone camera uses the entire D = 2 m array instead of just the sensors on the virtual 
aperture that the beamformer camera is restricted to, and the extra sensors lead to improved 
resolution. 
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Figure 6: Images of nearby objects formed from pure quasi light fields are blurry. In the scene, a 
small backlit pinhole moves across the field of view of a sensor array that implements three cameras, 
each computing one pixel value for each pinhole position, corresponding to a fixed surface patch. 
As the pinhole crosses the fixed scene surface patch, the near-zone camera resolves the pinhole 
down to its actual size of 1 mm, while the far-zone camera records a blur 66 cm wide. 



5.3 Generalized Beamforming 

We compare image formation from light fields with traditional perspectives on coherent 
image formation, by relating quasi light fields and our coherent cameras with the classic 
beamforming algorithm used in many coherent imaging applications, including ultrasound 
^ and radar [37]. The beamforming algorithm estimates a spherical wave diverging from 
a point of focus by delaying and averaging sensor measurements. When the radiation is 
narrowband, the delays are approximated by phase shifts. With the sensor array geometry 
from Section 15.21 the beamformer output is 

g = 'y^^T{md)U{md) exp(— zfcAm), (39) 

m 

where the T{md) are amplitude weights used to adjust the beamformer's performance. As 
moves into the far zone, 

- Ao ^ mds"^ mdsl, (40) 
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so that apart from a constant phase offset, (l39l) becomes a short-time Fourier transform 

g'^ = Y^ T{md)U{md) exp{-ikmdsl) . (41) 

m 

Evidently, \g°°\'^ is a spectrogram quasi hght field, and we may select T to be a narrow 
window about a point r to capture L^(r,s°). We have already seen how quasi light fields 
generalize the spectrogram. 

Beamformer applications instead typically select T to be a wide window to match the 
desired virtual aperture, and assign the corresponding pixel value to the output power \g\'^. 
We can decompose the three cameras in Section [5^ into such beamformers. First, we write 

in ( |38l) in terms of two different beamformers, 

Pp = 9l92, (42) 

where 

gi= U{nd) exp (-iA;A„) (43) 

\nd\<A/2 

and 

92 = YU (md) exp {-ikAm) , (44) 

m 

SO that the windows for gi and g2 are rectangular with widths matching the aperture A and 
sensor array D, respectively. Next, by construction 

Pf = \9i\'- (45) 
Finally, in the far zone, s*^ s" in fl37|) so that 

p''^{grr9T, (46) 

where g^ and g^ are given by ( HTj) with the windows T used in (1431) and (jBl). In other 
words, the near-zone camera is the Hermitian product of two different beamformers, and is 
equivalent to the far-zone camera in the far zone. 

We interpret the role of each component beamformer from the derivation of (!38|) . Beam- 
former gl aggregates power contributions across the aperture using measurements of the 
conjugate field U* on the aperture, while beamformer g2 isolates power from the point of 
focus using all available measurements of the field U. In this manner, the tasks of aggre- 
gating and isolating power contributions are cleanly divided between the two beamformers, 
and each beamformer uses the measurements from those sensors appropriate to its task. In 
contrast, the beamformer camera uses the same set of sensors for both the power aggregation 
and isolation tasks, thereby limiting its ability to optimize over both tasks. 

The near-zone camera achieves a new tradeoff between resolution and anisotropic sensi- 
tivity. We noted that the near-zone camera exhibits better resolution than the beamformer, 
for the same virtual aperture (Figure [6]). This is not an entirely fair comparison because the 
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near-zone camera is using sensor measurements outside the aperture, and indeed, a beam- 
former using the entire array would achieve comparable resolution. However, extending the 
aperture to the entire array results in a different image, as anisotropic responses are averaged 
over a wider aperture diameter. We interpret the near-zone camera's behavior by computing 

the magnitude 

K\ = vWbF- (47) 

Evidently, the pixel magnitude of the near-zone camera is the geometric mean of the two 
traditional beamformer output powers. |P^| has better resolution than |(yfip and better 
anisotropic sensitivity than |5'2p- 

Image formation with alternative light fields uses the conjugate field and field measure- 
ments to aggregate and isolate power in different ways. In general, image pixel values do not 
neatly factor into the product of beamformers, as they do with the Rihaczek. 

6 Concluding Remarks 

We enable the use of existing incoherent imaging tools for coherent imaging applications, by 
extending the light field to coherent radiation. We explain how to formulate, capture, and 
form images from quasi light fields. By synthesizing existing research in optics, quantum 
physics, and signal processing, we motivate quasi light fields, show how quasi hght fields ex- 
tend the traditional light field, and characterize the properties of different quasi light fields. 
We explain why capturing quasi light fields directly with intensity measurements is inher- 
ently limiting, and demonstrate via simulation how processing scalar field measurements in 
different ways leads to a rich set of energy localization tradeoffs. We show how coherent 
image formation using quasi light fields is complicated by an implicit far-zone (Fraunhofer) 
assumption and the fact that not all quasi light fields are constant along rays. We demon- 
strate via simulation that a pure light field representation is incapable of modeling near-zone 
diffraction effects, but that quasi light fields can be augmented with a distance parameter 
for greater near-zone imaging accuracy. We show how image formation using light fields 
generalizes the classic beamforming algorithm, allowing for new tradeoffs between resolution 
and anisotropic sensitivity. 

Although we have assumed perfectly coherent radiation, tools from partial coherence 
theory 1) allow us to generalize our results, and 2) provide an alternative perspective on 
image formation. First, our results extend to broadband radiation of any state of partial 
coherence by replacing U{r^)U*{r^) with the cross-spectral density Vr(r^, r^, u). W provides 
a statistical description of the radiation, indicating how light at two different positions, 
and r*", is correlated at each frequency u [38]. Second, W itself may be propagated along rays 
in an approximate asymptotic sense [391 SO], which forms the basis of an entirely different 
framework for using rays for image formation, using the cross-spectral density instead of the 
light field as the core representation. 

We present a model of coherent image formation that strikes a balance between utility 
and comprehensive predictive power. On the one hand, quasi light fields offer more options 
and tradeoffs than their traditional, incoherent counterpart. In this manner, the connection 
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between quasi light fields and quasi-probability distributions in quantum physics reminds us 
of the potential benefits of forgoing a single familiar tool in favor of a multitude of useful yet 
less familiar ones. On the other hand, compared with Maxwell's equations, quasi light fields 
are less versatile. Therefore, quasi light fields are attractive to researchers who desire more 
versatility than traditional energy-based methods, yet a more specialized model of image 
formation than Maxwell's equations. 

Quasi light fields illustrate the limitations of the simple definition of image formation 
ubiquitous in incoherent imaging. An image is the visualization of some underlying physical 
reality, and the energy emitted from a portion of a scene surface towards a virtual aperture 
is not a physically precise quantity when the radiation is coherent, according to classical 
electromagnetic wave theory. Perhaps a different image definition may prove more funda- 
mental for coherent imaging, or perhaps a quantum optics viewpoint is required for precision. 
Although we have borrowed the mathematics from quantum physics, our entire discussion 
has been classical. Yet if we introduce quantum optics and the particle nature of light, we 
may unambiguously speak of the probability that a photon emitted from a portion of a scene 
surface is intercepted by a virtual aperture. 
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